The present invention relates generally to digital data processing, and more particularly to the recovery of certain data in a digital computer system.
A modern computer system typically comprises a central processing unit (CPU) and supporting hardware necessary to store, retrieve and transfer information, such as communications busses and memory. It also includes hardware necessary to communicate with the outside world, such as input/output controllers or storage controllers, and devices attached thereto such as keyboards, monitors, tape drives, disk drives, communication lines coupled to a network, etc. The CPU is the heart of the system. It executes the instructions which comprise a computer program and directs the operation of the other system components.
From the standpoint of the computer""s hardware, most systems operate in fundamentally the same manner. Processors are capable of performing a limited set of very simple operations, such as arithmetic, logical comparisons, and movement of data from one location to another. But each operation is performed very quickly. Programs which direct a computer to perform massive numbers of these simple operations give the illusion that the computer is doing something sophisticated. What is perceived by the user as a new or improved capability of a computer system is made possible by performing essentially the same set of very simple operations, but doing it much faster. Therefore continuing improvements to computer systems require that these systems be made ever faster.
The overall speed of a computer system (also called the xe2x80x9cthroughputxe2x80x9d) maybe crudely measured as the number of operations performed per unit of time. Many improvements have been made and continue to be made to increase the speed of individual computer processors. However, there are certain limits to processor clock speed, number of circuits on a chip, and so forth which limit the overall throughput of a single processor. To support increasing demand for computing resource, it has become common in many large systems to employ multiple processors as a means of further increasing the throughput of the system. Additionally, such large systems may have multiple caches, buses, I/O drivers, storage devices and so forth.
The proliferation of system components introduces various architectural issues involved in managing these resources. For example, multiple processors typically share the same main memory (although each processor may have its own cache). If two processors have the capability to concurrently read and update the same data, there must be mechanisms to assure that each processor has authority to access the data, and that the resulting data is not gibberish. Another architectural issue is the allocation of processing resources to different tasks in an efficient and xe2x80x9cfairxe2x80x9d manner, i.e., one which allows all tasks to obtain reasonable access to system resources. There are further architectural issues, which need not be enumerated in great detail here.
One recent development in response to this increased system complexity is to support logical partitioning of the various resources of a large computer system. Conceptually, logical partitioning means that multiple discrete partitions are established, and the system resources of certain types are assigned to respective partitions. Specifically, processor resources of a multi-processor system may be partitioned by assigning different processors to different partitions, by sharing processors among some partitions and not others, by specifying the amount of processing resource measure available to each partition which is sharing a set of processors, and so forth. Each task executes within a logical partition, meaning that it can use only the resources assigned to that partition, and not resources assigned to other partitions.
Logical partitions are generally defined and allocated by a system administrator or user with similar authority. I.e., the allocation is performed by issuing commands to appropriate management software resident on the system, rather than by physical reconfiguration of hardware components. It is expected, and indeed one of the benefits of logical partitioning is, that the authorized user can re-allocate system resources in response to changing needs or improved understanding of system performance. Some logical partitioning systems support dynamic partitioning, i.e., the changing of certain resource definition parameters while the system is operational, without the need to shut down the system and re-initialize it.
One of the design goals of many large modern computer systems is availability. It is desirable that these systems be available to users as much as possible. In some cases, there is a need for constant availability, i.e., the system must be designed so that it is always available, come what may. In other systems, some amount of down time, or some amount of time when the system operates at reduces performance, may be acceptable.
In general, there is some trade-off between availability and maximum utilization of hardware resources for productive work. For example, it is well known to store data in a redundant fashion on multiple storage devices in any of various schemes known as xe2x80x9cRAIDxe2x80x9d, meaning xe2x80x9credundant array of independent disksxe2x80x9d. However, all of these schemes sacrifice some of the storage capacity of the disks in order to achieve redundancy. Additionally, these schemes may adversely affect storage access times when compared with non-redundant storage schemes.
Among system availability tools are certain functions which handle the recovery of compiled data. As used herein, compiled data is any data which is derived or derivable from other data stored in the system. Basic non-compiled, or raw, data, is not necessarily recoverable by any deterministic system process if the data becomes lost due to system failure or other reason. Unlike this raw data, compiled data, if lost, can always be recovered by deriving it again from the raw data on which it was based. However, re-deriving the compiled data may take a very long time, during which the compiled data may be unavailable or the performance of the system adversely affected.
One common type of compiled data is a database index. Large computer systems often support very large databases. Information may be selectively extracted from such databases by means of various database queries. Since these queries can consume significant processor resources, indexes of data are established to support queries of specific fields in the database. An index is typically associated with a specific field in the database records, and orders the records in the database according to the value in that field. For example, the index may be an ordered list of pointers to database records, where the pointers are ordered according to a field value. Alternatively, the index may be an ordered list of value and pointer pairs, where the value is the value in the associated field, and the pointer a pointer to the database record, the ordered list being sorted according to field value.
The time required to regenerate compiled data can be shortened or eliminated if the compiled data is saved in certain recovery assistant forms as changes are made. However, such techniques generally have associated costs in terms of additional hardware and/or system performance degradation. As an extreme example, compiled data can be saved to storage in the same manner as raw data, so that it is simply read from storage or otherwise recovered in the same manner as raw data, without lengthy regeneration. This naturally increases the number of I/O operations. A less intrusive technique is logging or journalling in some fashion as changes are made to the raw data which affect the compiled data. Depending on the number of objects logged, the frequency of logging, and so forth, a longer or shorter recovery time may be experienced following a failure, the less intensive logging techniques typically requiring a longer recovery period following failure. Thus, in general it may be said that there is a design trade-off for a given system between maximum performance during normal operations and time to recover compiled data in the event of a failure.
One existing data protection utility which supports the maintenance of compiled data for regeneration, and specifically supports the maintenance of database indexes for regeneration, in the event such data is lost due to some event, such as a power failure, is disclosed in U.S. Pat. No. 5,574,897 to Hermsmeier et al. and U.S. Pat. No. 5,625,820 to Hermsmeier et al. A system may selectively log certain database indexes, in accordance with a user specified recovery time. If the user specifies that the system must respond with a relatively short recovery time, more intensive logging is generally required; if the user specifies that the system can tolerate a relatively longer recovery time, the system can reduce the logging activity.
Where a system is logically partitioned, a data protection utility executes in one of the logical partitions. In the case of dynamically defined logical partitions, it is possible that the parameters of the logical partition in which the data protection utility executes will change. The utility may therefore over-estimate or under-estimate the recovery time required in a dynamically logically partitioned environment. In the case of an over-estimate, the utility may perform excessive or unnecessary redundancy functions during normal system operation, resulting in a decline in system performance. In the case of an under-estimate, the utility may fail to recover in time following a failure. A need therefore exists, not necessarily recognized, to assure that recovery strategies accurately reflect the current system configuration under which they are to be executed.
A protection utility for compiled data in a computer system having dynamically configurable logical partitions determines the time for rebuilding compiled data, and selectively stores data in a form not requiring rebuild in order to meet a pre-specified recovery time limit. If the configuration changes, the protection strategy is migrated to adapt to the new configuration.
In the preferred embodiment, the compiled data is multiple database indexes for one or more potentially large databases. The user specifies a maximum recovery time for the database indexes. The protection utility automatically calculates the recovery time for each index, using the current configuration of the logical partition in which the protection utility executes. If the total recovery time is more than the specified maximum, at least some of the indexes are logged to reduce the recovery time. If the configuration of the logical partition is changed so that partition resources are reduced, requiring a longer recovery time, the protection utility determines the new recovery time, and begins to log additional indexes as changes are made to migrate the recovery time below the specified maximum. If the configuration of the logical partition is changed so that partition resources are increased, requiring a shorter recovery time, the protection utility selects indexes for discontinued logging, so that recovery time migrates upward toward the limit.
In the preferred embodiment, the system is configured as a plurality of logical partitions, each partition having an assigned set of physical processors (which may be shared with one or more other partitions), an assigned processor resource measure (i.e., a processor resource measure in units of equivalent physical processors) and an assigned number of virtual processors. In a rough sense, each partition behaves as if it contains as many processors as the assigned number of virtual processors, each such processor having an appropriate fraction of the processing capacity of a physical processor. The set of physical processors, the processor resource measure, and the number of virtual processors assigned to a logical partition may be dynamically altered during system operation. The processor resource measure is the primary indicator of recovery time used by the recovery tool.
By migrating the recovery times to the specified target times in an environment in which the system may be dynamically changing, the system avoids unnecessary I/O operations relating to recovery, and avoids excessive recovery times.
The details of the present invention, both as to its structure and operation, can best be understood in reference to the accompanying drawings, in which like reference numerals refer to like parts, and in which: